Sparse Online Variational Bayesian Regression

This work considers variational Bayesian inference as an inexpensive and scalable alternative to a fully approach in the context of sparsity-promoting priors. In particular, priors considered arise from scale mixtures Normal distributions with generalized inverse Gaussian mixing distribution. includes LASSO introduced [65]. It also family which more strongly promote sparsity. For linear models method requires only iterative solution deterministic least squares problems. Furthermore, for p unknown covariates can be implemented exactly online cost $O(p^3)$ computation $O(p^2)$ memory per iteration -- other words, is independent n, principle infinite data considered. large $p$ approximation able achieve promising results $O(p)$ iteration, both memory. Strategies hyper-parameter tuning are The real simulated data. shown that performance terms variable selection uncertainty quantification comparable problems tractable method, fraction cost. present comfortably handles $n = 65536$, $p 131073$ on laptop less than 30 minutes, 10^5$, 2.1 \times 10^6$ overnight.